The present invention relates to a moving picture processing apparatus for use in television broadcast program production, video program production or video editing, to a video processing method and apparatus which can be applied to a moving picture searching apparatus for search of a video source material library, etc. as well as to a computer program product having a video processing procedure recorded therein. More particularly, the present invention pertains to a computer-aided moving picture editing apparatus which can facilitate confirmation of contents of a moving picture source material during editing of the moving picture material and thus can shorten its editing time, by displaying an icon indicative of the picture material to be edited in a window on a display screen and operating the icon on the window.
As computers advance in processing speed and function, software has been quickly diversified and enhanced in their function. A facility for making a television broadcast program or a video program has also advanced in its function by applying such advance in computers and software to the facility.
In particular, editing of image information in making a program has been recently carried out with use of, e.g., a moving picture editing apparatus having a randomly-accessible information memory device using an inexpensive hard disk easy to handle and save. By employing such high-performance, diversified computer and software in the moving picture editing apparatus, the apparatus will be increasingly enhanced in its function.
FIG. 3 is a block diagram of an exemplary arrangement of a moving picture editing apparatus which embodies a moving picture editing method with the aid of a computer.
The illustrated moving picture editing apparatus includes a central processing unit (CPU) 301 for performing various sorts of control and editing operations on respective constituent devices of the moving picture editing apparatus under control of software programs; a display unit 308 for displaying control information on the CPU 301 as well as information on editing of motion images, scene images, cut images and a tree structure indicative of an editing status; an input device 309 for selecting information displayed on the screen of the display unit 308 and inputting a command to the CPU 301; a frame buffer memory 307 for storing therein picture information to be displayed on the display unit 308; and a picture reproducer (e.g., video tape recorder) 305 for reproducing a video signal for a moving picture for use in the editing and outputting the reproduced video signal to a video interface 304. The video interface 304 functions to convert the video signal of the moving picture received from the picture reproducer 305 into video information having such a format as able to be processed in the apparatus and to output the video information to an information memory device 306, a cut image change point detector 303 and a memory 302. The video interface 304 also functions to convert the picture information having such a format as able to be processed in the apparatus and received from the information memory device 306 into the video signal and to output the video signal from a video signal output terminal 311. The cut image change point detector 303 functions to judge that adjacent frame images of the moving picture belong to different cut images, by comparing the adjacent frame images and determining that an image information value indicative of a difference between the frame images exceeds a predetermined level. The memory 302 functions to store therein various sorts of software control programs for the CPU 301 and also to temporarily store various types of control information (such as a series of frame numbers of the moving picture, a frame number for the detected cut-image change point, a reduced image (M-icon) to be displayed on the screen of the display unit 308, successive numbers of cut images, a sequence of numbers indicative of an order of a plurality of cut images) to be used in the apparatus. The information memory device 306 of a random access memory type functions to store information about the editing moving picture which is reproduced by the picture reproducer 305 and which has the predetermined format converted by the video interface 304. The information memory device 306 also functions to store various editing and control information (such as M-icons) which are temporarily stored in the memory 302 to be used in the apparatus. The information memory device 306 uses, e.g., a hard disk of a random access memory type. The apparatus further includes a bus 310 for digital information transmission between the constituent elements of the moving picture editing apparatus. Although the various control programs of the CPU 301 have been explained in connection with a case where the programs are stored in the memory 302, the programs may be stored in a memory part (not shown) built in the CPU 301.
Connected to the bus 310 are the CPU 301, memory 302, cut image change point detector 303, video interface 304, information memory device 306 (e.g. a magnetic memory device), frame buffer memory 307, display unit 308, and input device 309 as the constituent elements of the moving picture editing apparatus. The CPU 301 transmits various types of signals between the constituent elements. These elements are controlled by an access signal issued from the CPU 301. An instruction to the CPU 301 is issued from such an input device 309 as a mouse or a keyboard.
When the moving picture editing apparatus is arranged as mentioned above and predetermined software programs are stored in the memory 302 (or the CPU 301), information on the moving picture to be edited can be stored in the memory 302 and information memory device 306 for its management. Further, when the moving picture stored in the memory 302 and information memory device 306 is displayed on the screen of the display unit 308 as cut images and scene images, and when these cut and scene images are used to form a tree structure; an operator can edit the moving picture.
A moving picture source material includes a plurality of sets of "cut" images. The cut image is defined as a set of a series of frame images taken in one shooting operation by a camera. Data on representative one of the frame images in each cut image is thinned out to form a reduced size image, which image will be referred to as M-icon. By operating the displayed M-icon on the display screen with use of a pointing device, the operator can carry out such editing operations as move, delete, couple, transform or special effect on the entire "cut" represented by the M-icon. One of methods of dividing a video source material into a plurality of cut images is, for example, to analyze features of such video data as illuminance, sound volume or color tone of a moving picture, to recognize as a cut boundary a remarkable change in these features and to automatically divide the moving picture into cut images sequentially at such recognized cut boundaries. This method is disclosed, e.g., in JP-A-2-184181 Publication.
A "scene" image includes a plurality of cut images. Further a plurality of scene images having a specific "shooting theme" form an editing moving picture. The editing moving picture is created by combining a tree structure having the cut images in a lower layer and the scene images with the "shooting theme" in an upper layer with M-icons of the video material.
FIG. 4 is an example of a display screen when a moving picture is edited with use of the moving picture editing apparatus of FIG. 3 using a moving picture editing application program.
The display screen has a window 100, which in turn includes an editing area 105 and a material area 106.
Displayed in the material area 106 as registered and arranged as editing materials are M-icons 101, 102, . . . and 104 which indicate as representative images the first frame images of cut images judged and divided at each cut image change point by the cut image change point detector 303. Although each M-icon is illustrated as a sheet of representative image for simplicity in the drawing, the M-icon is actually as illustrated 3-dimensionally on the display screen in the form of stacked sheets of frame provided with a thickness corresponding to the number of sheets.
Also displayed in the editing area 105 is a tree structure 107 of a combination of the M-icons of the edited moving picture. That is, M-icons edited and registered are displayed at a predetermined position in the tree structure 107.
Editing of the moving picture is carried out, for example, by placing a cursor or pointer on desired one of the M-icons 101, 102, . . . and 104 in the material area 106 with use of the mouse as the input device 309 to select the desired M-icon, and then dragging the desired M-icon to a desired position in the tree structure of the editing area 105 with the mouse button kept in its clicked state, thereby building up the M-icon as the typical cut image into a tree structure.
However, when the moving picture editing is carried out in this manner, in order to select desired one from the M-icons 101, 102, . . . and 104 in the material area 106 and drag it to the desired position in the tree structure 107 of the editing area 105 for its registration; the operator must confirm the contents of the selected editing material (desired M-icon). For example, the operator must examine the cut images for the M-icon from its beginning to end to confirm that the cut images have the same contents, e.g., all contain the same person or that the cut images contain different contents, e.g., person and scene are mixed.
Conventionally, in order to confirm the contents of the desired M-icon in this manner, the operator must open another preview window on the screen of the display unit 308, and actually preview the cut image data of the desired M-icon to confirm the contents of the cut images from its beginning to end, which requires a lot of time.
A moving picture is edited by selecting desired one from a plurality of M-icons displayed in a material area of a window, and then moving the desired M-icon to a desired position in a tree structure in an editing area to build up such M-icons into a tree structure. In this case, an editor or operator is required to confirm that image data of the selected M-icon have such contents as to meet operator's demand. However, a prior art moving picture editing method has had a problem that the method requires a lot of time to confirm the contents of the image data of the selected desired M-icon, because the operator must preview the image data contents from the beginning to the end in a normal reproduction speed or other reproduction speed close to the normal speed.